In order to reduce the bit-rate of the encoder output, a scalable bit-stream may comprise a form of inter-layer prediction. Exemplary systems comprise inter-layer prediction within the scalable video extensions for the AVC|H.264 video coding standards. These extensions are commonly known as SVC, and the SVC system, described in T. Wiegand, G. Sullivan, J. Reichel, H. Schwarz and M. Wien, “Joint Draft 9 of SVC amendment (revision 2)”, JVT-V201, Marrakech, Morocco, Jan. 13-19, 2007, which is hereby incorporated herein by reference. In the SVC system, inter-layer prediction is realized by projecting motion and mode information from an enumerated lower layer to an enumerated higher layer. In addition, prediction residual is projected from an enumerated lower layer to an enumerated higher layer. The higher layer bit-stream may then contain additional residual to improve the quality of the decoded output.